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Counting the number of ground states for a spin-glass or NP-complete combinatorial optimization 
problem is even more difficult than the already hard task of finding a single ground state. In 
this paper the entropy of minimum vertex-covers of random graphs is estimated through a set of 
iterative equations based on the cavity method of statistical mechanics. During the iteration both the 
cavity entropy contributions and cavity magnetizations for each vertex are updated. This approach 
overcomes the difficulty of iterative divergence encountered in the zero temperature first-step replica- 
symmetry-breaking (IRSB) spin-glass theory. It is still applicable when the IRSB mean-field theory 
is no longer stable. The method can be extended to compute the entropies of ground-states and 
metastable minimal-energy states for other random-graph spin-glass systems. 
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A combinatorial optimization (CO) problem is de- 
fined by an energy function E{a) on configurations a — 
(cti, (T2, . . . , CTAf) of a iV-dimensional space, where each 
variable has only a finite number of states (e.g., 
Gi = ±1). For CO problems in the non-deterministic 
polynomial-complete (NP-complete) class, searching for 
configurations of energy equal or very close to the lowest 
possible value is in general a very difficult task. Statis- 
tical physicists relate this computational hardness to the 
emergence of complex structures in the problem's config- 
uration space and the proliferation of metastable macro- 
scopic states (macrostates) . The entropy spectrum (num- 
ber of configurations at each minimal-energy level) gives 
a characterization of the energy landscape of a hard CO 
problem. Using the first-step replica-symmetry-breaking 
(IRSB) cavity method of spin-glass theory [J 0,0], the 
ground-state energy densities for several hard CO prob- 
lems on random graphs have been calculated with high 
precision (see, e.g., Refs. 0, 0, Q)- estimating the 
ground-state entropy and the entropies of metastable 
states is still a challenging theoretical and computational 
issue. 

In the IRSB cavity approach, the ground-state energy 
of a hard CO problem is evaluated by first assuming 
the configuration space of the system can be clustered 
into many macrostates. A variable i experiences in each 
macrostate an integer- valued field hi. The distribution 
of this field among all the macrostates, Pi{hi), is then 
obtained through an iterative numerical scheme 0. To 
calculate the ground-state entropy, a conventional tech- 
nique is to introduce a temperature T and expand the 
field hi to first order in T at the limit T ^ ^] : 



hi = 



Tr,, 



(1) 



with rrii being an integer and a finite real value. A set 
of iterative equations are derived to obtain the joint dis- 
tribution Pi{mi, Ti) of the values and for each vari- 
able i. The ground-state entropy S'^ is then the first or- 
der term in the free energy expansion F{T) = — TS'^. 



This approach works for some relatively simple problems 
(e.g., graph-matching whose configuration space is er- 
godic [6|] , random Q-coloring and if-SAT which have zero 
ground-state energy 0,11,0) but it fails for many other 
NP-complete CO problems, for which the ground-state 
energy is positive and the T — > IRSB cavity equa- 
tions are not stable. For these later systems, at T = the 
field hi is not necessarily an integer and the correction 
in Eq. ([T]) usually diverges at the T ^ limit. Conse- 
quently, although the T ^ IRSB cavity method is able 
to estimate the ground-state energy of a hard CO prob- 
lem with high accuracy (as only the value of m; but not 
that of ri is used) , it often reports a negative or divergent 
ground-state entropy. 

In this paper we use a different way to estimate 
the ground-state entropy of a CO problem or a finite- 
connectivity spin-glass. We work directly at temperature 
zero and, within the IRSB cavity framework, calculate 
both the cavity magnetizations and cavity entropies of 
each variable in each macrostate. A very small cutoff 
e is naturally introduced in the iteration of cavity mag- 
netizations to avoid divergence in the iteration of cav- 
ity entropies. This approach gives good results when 
tested on the random vertex-cover problem, the random 
3-SAT problem, and the random ±J spin-glass model, 
even when the T ^ IRSB mean-field spin-glass theory 
is no longer stable. It is also able to calculate the en- 
tropies for other minimal-energy levels. Here we focus on 
the vertex-cover problem (a prototypical NP-hard prob- 
lem [l3|) to demonstrate the main ideas of this method. 
Detailed calculations on other model systems will be re- 
ported elsewhere. 

Let us first briefly introduce the vertex-cover problem 
L. IT^. A graph Q contains N vertices and M edges. 
Each edge is between a pair of vertices The 

mean connectivity of the graph is c = 2M/N, which is 
the number of edges a vertex on average is connected 
to. A vertex cover C for graph Q contains a subset 
of vertices of Q such that for each edge (i,j) € Q, at 



2 



least one of its extremities is in C. If a vertex i is con- 
tained in a vertex-cover C, we say it is covered in C. 
The energy of a vertex-cover C is defined as its car- 
dinality, E{C) = \C\. Graph Q has many vertex cov- 
ers, with energy ranging from the maximum value N to 
the minimum value Eg = mine \ We denote by Cg 
the set of minimum vertex covers (MVCs) of graph Q: 
= {C : \C\ = Eg}. To determine exactly the energy 
i?" of MVCs for a graph in general is a very hard compu- 
tational problem; to count the number of MVCs is even 
harder. Using the cavity method of statistical mechan- 
ics, the mean energy density of MVCs for random graphs 
has been evaluated in Refs. [1, 



11, 1 



The entropy of 
MVCs for a random graph was also estimated by com- 



puter simulations [I4I . This work complements Ref . [12 1 
by giving an analytical estimation of the ground-state 
entropy ^0 = log |C0 1 . 

We denote by TVi the probability of a vertex i being 
contained in the MVCs, 



1 



(2) 



where I is the indicator function. Similar to tt^, for a 
vertex j £ di (the set of vertices which are connected to 
i by an edge), we denote by TTj^^ its probability of being 
in the MVCs of the cavity graph Q\i (which is obtained 
from Q by removing vertex i and all its edges). 

We consider here the ensemble of random graphs with 
mean connectivity c. When the graph size N is suffi- 
ciently large the length of a loop in the graph is of order 
log^{N), and a random graph Q is locally tree-like. Con- 
sider the set di of vertices in the neighborhood of a ran- 
domly chosen vertex i. If the edges between i and these 
vertices are deleted, in the cavity graph Q\i the short- 
est length between any two vertices j, k ^ di diverges 
logarithmically with N. As a first step, it is therefore as- 
sumed that these vertices are uncorrelated in the cavity 
graph G\i, and the probability of finding these vertices 
in a MVC of Q\i can be expressed in a factorized form: 



g\ij 



n 



(3) 



Equation Q is called the Bethe-Peierls approximation 
and the corresponding cavity method is referred to be 
replica-symmetric (RS). Under this assumption, the en- 
ergy and entropy of MVCs for a random graph Q can be 
expressed as [J] 



El = ^A£;,-^A%,) , 



(4) 
(5) 



where AE'a, and AS'x are, respectively, the contribution 
to the system's ground-state energy and entropy from an 



vertex (x = i) or an edge {x = (i,j)). For example, 
AE, = El- El^^ and AS, = log [Cg 1 - log IC^^J. 

We shall distinguish three situations when writing 
down the expressions for AEi and A^^. The total num- 
ber of MVCs of cavity graph Q\i which contain the set 
di is HjeOi ''''ili- This number is positive if all the 

cavity probabilities iTj^i > 0. In this case, when ver- 
tex i is added, it will not be in any MVC of Q but the 
set di will be contained in every MVC of Q. Then we 
have TTi = (vertex i is always non-covered), AEi — 0, 
and \Cl\ — |Cg^J Hjeai ""il*' ^^'^ ^^'^ change in entropy is 
ASi = logTTjii. On the other hand, if in the set di 
there are at least two vertices which are not in any MVCs 
of cavity graph G\i-, then vertex i will be present in all the 
MVCs of graph G- Adding vertex i to a MVC of cavity 
graph g\i results in a MVC of graph G (|Cg| = |C°^J). 
In this case tt^ = 1 (i is always covered), AEi = 1, and 
AS*, = 0. 

Now we consider the remaining case, namely in the set 
di there is only one vertex (say j) which is not in any 
MVC of graph G\i- In this case, each MVC of graph G 
contains either vertex i or vertex j but not both. The 
energy increase is AEi = 1, and the total number of 
MVCs for graph G is 



\cl\ 



\C? 



\C? 



evji n '^^h 
keai\j 



(6) 



where Cgy^ j is the complete set of MVCs for the cav- 
ity graph G\i,j (the remaining graph after further re- 
moving vertex j from G\i), and di\j is the set of di- 
rect neighbors except j of vertex i. The first term on 
the right hand side of the equality in Eq. ([6]) is the 
number of MVCs which contain vertex i, while the sec- 
ond term is the number of MVCs which contain j. In 
this case vertex i is said to be unfrozen. The entropy 
change AS, = log(l + e^'^'^^i* n^GSiVj '^'^lO ' ^^'^ ^'^ = 



1/(1 



-AS,- 



n 



k£di\j 



where AS". 



log IC 



log |C^^ ■ ■ I is the entropy change due to adding vertex j 
to the cavity graph G\i,j- We refer ASj^i as a cavity 
entropy of vertex j. Notice that in this case, to prop- 
erly calculate ASi, one needs not only to consider all the 
MVCs of cavity graph G\i but also vertex-covers with a 
higher energy Eg^^ - + 1. 

The energy and entropy contribution of an edge (z,j) 
can also be obtained similarly. If at least one of the cavity 
probabilities nj^i and iTiy is positive, then Ai?(j j) — 
and AS'(ij) = log(l — (1 — ~ On the other 

hand, if both iTi^j and nj^i arc zero, then AE(^i j) = 1 and 

A5(,,,) =log(e-^^-i. +e~^^.i-). 

On each of the 2M directed edges j ^ i of graph G 
there is a cavity probability TTj\i and a cavity entropy 
ASj\i. After all these AM values are determined, the 
total ground-state energy and entropy can then be eval- 
uated using Eqs. dH) and ([5]). Starting from a random 
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FIG. 1: (Color Online) Ground-state entropy density of the 
vertex-cover problem as a function of the mean vertex degree c 
of the random graph. Circles are simulation data [ij], up- and 
down-triangles are, respectively, theoretical results obtained 
by the replica-symmetric and the IRSB cavity method. The 
finite-temperature entropy density values at T = 0.091 and 
0.04 are also shown as a comparison. 

initial condition, the values of 7Tj\i, ^Sj\i, and the cavity 
energy increase AE'jij can be determined by the following 
iterative algorithm. Consider vertices k in the neighbor- 
hood dj\i of a vertex j: 

(1) If aU nk\j > 0, then tt^i, = 0, AE'^j, = 0, ASj\, = 
J2kedj\z logT^fcij- 

(2) If two or more ir^j — 0, then vrjjj = 1, AEj^^ ~ 1, 
AS,\, = 0. 

(3) If only one Tr^jj = 0, then AEj^i = 1, and 

ledj\i,k 

AS,i, = log(l + e-^'^-i^ n '^'b) • (8) 

ledj\i,k 

When using Eq. ([7|) to update the cavity probability 7rj|,j, 
sometimes iTj^i becomes extremely close to zero. A cutoff 
e is therefore introduced in the numerical scheme: If the 
obtained value of Tr^ij is less than e we set Tr^ij = 0. The 
theoretical entropy values shown in Fig. [1] are obtained 
by setting a cutoff value of e = 10"^° (more discussions 
on the choice of e are mentioned below). 

The above discussion concerns with one large graph Q. 
As we are interested in the graph-averaged values for the 
ground-state energy and entropy, a population dynamics 
simulation can be performed similarly by storing a large 
array of TTjii and ASj\i and then updating this array 
The ground-state entropy density s° = limjv_»oo Sg/N as 
a function of mean connectivity c of the random graph 
Q is evaluated by this numerical scheme (Fig. (TJ up- 
triangles). The ground-state entropy first increases with 
c and reaches a maximum at c w 1 , then it decreases with 
c. For c < 2.7183 the RS cavity method is known to be 



valid [3, lisl and it correctly predicts the ground-state en- 
tropy and energy density for the system. For c > 2.7183 
however, the RS prediction is systematically lower than 
the simulation result of Hartmann and Weigt (circles in 
Fig. [T]) and even becomes negative for c > 4.4 (such an 
entropy crisis was also observed in the hitting set prob- 
lem [5|). For c > 2.7183 the entropy predicted by the 
RS cavity method depends strongly on the cutoff e. If a 
smaller e value is used, the entropy decreases even faster 
with the mean connectivity c. 

When the mean connectivity c of the random graph 
is larger than 2.7183, the Bethe-Peierls approximation 
Eq. ([3]) is no longer a good assumption, as there are 
strong long-range correlations among distant vertices 
(l3| . For example, consider two vertices i and j whose 
shortest-path length is of order log,, N and suppose these 
two vertices both are unfrozen (tt^ > 0, tt^ > 0) among 
the space of MVCs of graph Q. The Bethe-Peierls ap- 
proximation assumes that the probability of finding a 
MVC which contains both i and j is equal to iTiTTj > 0. 
However, it may be the case that there is not a single 
MVC in which both i and j are present [l^3. To take 
into account such long-range correlations, in the IRSB 
cavity theory the MVC set Cg of a graph Q is clustered 
into many subsets " which are indexed by an index a. 
In each such subset it is assumed that the Bethe-Peierls 
approximation still holds, and Eq. ([3|) is replaced by 

P(9zcC:CeC°-C)=n^."»- (9) 

jei 

where tt^^ is the probability of vertex j being in the 
MVCs of the a-th subset Cg'^^ of the cavity graph Q\i. 
Because of Eq. (|9]), the iterative equations for Wj^i and 
ASj\i as mentioned before are still valid in each subset of 
MVCs. To characterize the property of different clusters, 
a probability distribution 'Pj\i(TT, s) is introduced on each 
directed edge of the graph, which is equal to the frac- 
tion of MVC clusters with tTjIj — it and ASj\i = s. For 
a single graph Q these 2M probability distributions 'Pj\i 
can again be obtained by iterations, and the correspond- 
ing distribution of Vj\i{TT,s) among all the edges of the 
random graph can be obtained by mean-field population 
dynamics. 

The iteration equation for Vj\i{'K, s) reads 




e-y'^^^\^5{^-^,\i)5{s-AS,\,) . (10) 

The value of the re-weighting parameter y in the above 
equation is chosen such that a complexity parameter is 
equal to zero 0. In writing down Eq. (jlOp . it is further 
assumed that the joint probability of observing the cavity 
values {iTi^^j, ASk\j), (7r;|j , ASi\j), . . . for vertices k,l, . . . £ 
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FIG. 2: Predicted mean ground-state entropy of the random 
vertex-cover problem at connectivity c = 10 as a function of 
the cutoff parameter e. 



dj\i can be written in a factorized form: 

P(7rfci,,A^fc|,;...) = n nn|jK„|j,AS^|,) . (11) 

m^dj\i 

Details of the IRSB numerical iteration scheme for the 
vertex-cover problem can be found in Ref. 

The mean entropy density of MVCs for random graphs 
of mean connectivity c as obtained by this IRSB cavity 
method is shown in Fig. [1] (down-triangles). The theo- 
retical pred ictions are in good agreement with simulation 
results [l^l ■ The mean ground-state energy density as ob- 
tained by the present method is also in good agreement 
with the simulation and theoretical results of Ref. 4] . In 
the calculation we have used a cutoff value e — lO^^*'. 
Such a cutoff is necessary for the random vertex-cover 
problem, as the IRSB cavity approach is not stable to 
further steps of replica-symmetry- breaking lj|. Figure [2] 
shows that, the predicted value of the mean ground-state 
entropy density is not sensitive to the cutoff parameter e 
when e < 10^^. 

We also carry out lengthy population dynamics sim- 
ulations at finite temperatures using different protocols. 
The entropy values obtained at T = 0.091 and T = 0.04 
are shown in Fig. [TJ At these low temperatures al- 
though the obtained energy density values are almost in- 
discernible from the ground-state values, there is still a 
gap between the finite-temperature and the ground-state 
entropy density when c > 2.7183. If further lowering the 
temperature, the quality of the simulation results deteri- 
orate, possibly because of insufficient population size and 
insufficient equilibrium and sampling times. We were also 
unable to remove this gap by using instead the expan- 
sion Eq. ([1]), because the population dynamics diverges. 
In comparison with these, the zero-temperature direct 
method is computationally much efficient and also easier 
to implement. 

In summary, we have calculated the ground-state en- 
tropy for the random vertex-cover problem using the 



IRSB cavity approach of spin-glass theory. In our 
method, both the cavity probabilities ttj^^ and cavity en- 
tropies ASjii of each vertex j in a cluster of MVC so- 
lutions are recorded. We have paid special attention on 
unfrozen vertices (each of which belongs to some but not 
all MVCs of the graph) . As demonstrated by Eq. ^ , the 
entropy contribution of an unfrozen i comes not only from 
the MVCs of the graph Q\i but also form other higher- 
energy configurations of Similarly the cavity entropy 
AS'j|i of a vertex j also has two sources of contributions. 
Equation ([5]) is rather simple for the vertex-cover prob- 
lem, while for some other NP-hard CO problems and 
spin-glass models (e.g., the random-graph ± J spin-glass) 
counting the entropy contribution of an unfrozen vertex 
can be more complicated. A cutoff parameter e is intro- 
duced so that if tTjIj < e in one cluster, it is set to be zero. 
With this cutoff parameter, the present cavity method 
can still give good estimations on the ground-state en- 
tropy of a hard CO problem or spin-glass system even if 
more steps of replica-symmetry-breaking are needed to 
fully describe the system. 
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